What Peter Thiel gets wrong about existential risk.
Peter Thiel — tech billionaire, libertarian polemicist, Trump donor — recently gave a speech at the Oxford Union, one of the oldest and most prestigious student debating societies in the world, to kick off its 200th year. That’s hardly news — we’ve all heard Thiel’s spiel many times before on campus conformity and how only tech can save us.
But my ears pricked up this time as he specifically criticized my field. I’m an existential risk researcher at Cambridge University, where my colleagues and I study the risks from nuclear and biological weapons, climate change, and emerging technology such as synthetic biology and artificial intelligence. All of these technologies pose incredibly high risks — we think it’s plausible that one or more of them could lead to civilizational collapse or extinction, affecting everyone alive today. As many in the effective altruism community have argued, I think tackling these risks is a key priority of our time.
Thiel seems to have had a passing interest in these topics a decade ago, speaking at some conferences and donating some money. But to my knowledge he has not engaged with the existential risk reduction community for as long as I have been involved. Instead, he seems increasingly interested in seasteading and the alt-right.
So why was he criticizing the field of existential risk reduction? Thiel seems to suggest we in the community are Luddites, bearing some responsibility for the stagnation in real wages and technological progress since the 1970s. He claims a leading cause of stagnation is that scientists effectively have become too scared of their own technology. He told the Oxford Union audience that “the single answer as to why it is stalled out on the part of the universities is something like science and technology are just too dangerous.”
Existential risk isn’t a hoax
I don’t want to minimize the situation. It really is true that real wages for many workers in the UK and US have been stagnant since the 1970s, especially through the last grueling decade of austerity. And too much technical effort and venture capital has been suboptimally invested into e-commerce (like PayPal), social media and online advertising (like Facebook), or surveillance (like Palantir). (What’s the connection? All three companies helped to make Thiel’s estimated $8 billion fortune.)
But as someone who spends a fair amount of time encouraging technologists to consider their responsibilities for the technology they create, let me say that an overabundance of fear is not typically what I encounter.
Powerful technologies are often “dual-use”: They can be used to help or to harm people. Take nuclear physics. Nuclear weapons are in many ways the original existential risk, the one that has loomed over the world since 1945. However, nuclear power is also a reliable, zero-carbon source of power.
Advances in biotechnology enabled the quickest vaccine rollout in history, as well as further medical breakthroughs. But they also enable “gain-of-function” experiments where scientists purposely try to make diseases more virulent.
Language models like ChatGPT are wonderful and amazing but in the wrong hands could enable the mass production of disinformation, as we warned about in “The Malicious Use of Artificial Intelligence” in 2018. The problem with arguing for “more speed” is that these dual-use technologies are already moving faster than we can keep up.
Thiel is simply wrong if he thinks that “slow it down” is the only response. The Founders Pledge Climate Change Fund, a community of entrepreneurs who pledge to donate a portion of their exit earnings to charity, tries to speed up innovation in low-carbon concrete and steel. Alvea is a biotech startup aiming to speed up vaccine production. Anthropic is an AI safety company trying to speed up interpretability and alignment. We call this ”differential technology development”: speeding up safe or defensive tech relative to harmful or offensive tech.
But it’s also clear that the old Facebook motto of “move fast and break things” won’t work. Just speeding up technology won’t be enough to keep us safe. We need sensible domestic regulation: supporting the green transition, raising safety requirements in biological labs, and ensuring that high-risk AI systems go through safety tests. We need international agreements, like the Paris agreement and the nuclear arms control treaties that Donald Trump — whose presidential campaign Thiel donated to — ripped up.
A libertarian approach to X-risk doesn’t work
But Thiel doesn’t seem to want this. He’s an Ayn Rand libertarian. On ideological grounds, he doesn’t believe that government action can help. He thinks regulation makes things worse. When asked by an Oxford Union audience member about how he “would fix” the UK’s National Health Service (NHS), he said we need to get over our “Stockholm syndrome” as a country and privatize it already. Everyone is entitled to their own opinions, even billionaires. But it’s a fringe ideology. And it’s one that could do a huge amount of harm.
Thiel seemingly adopts this fend-for-yourself mentality in his planning too. He has long had a bolt-hole that he could escape to in case of societal collapse. In 2015, he bought a 193-acre plot of land (bigger than the Disneyland theme park in California) on the South Island of Aotearoa New Zealand for a reported $13.5 million. In May 2022, he was denied planning permission to build a luxury lodge on the plot with space for 24 guests, a theater lounge, a spa, and a “meditation pod.”
A mate of mine was traveling around the South Island a few years ago and had a pint at a pub. One of the locals pointed out a spot on a hill: “See there? That’s Peter Thiel’s house. If anything goes bad, that’s where we’ll go for food and water.”
It is a fantasy to think that existential risk can be reduced — or survived — just with individual action, the invisible hand of the market, and a “go faster” sign. It also needs collective action, wisdom and patience, and sensible and proportionate regulation.
That’s the approach that the existential risk and effective altruism communities are taking. But that, unfortunately, is the approach that Thiel appears to disagree with.
Haydn Belfield has been academic project manager at the University of Cambridge’s Centre for the Study of Existential Risk (CSER) for the past six years. He is also an associate fellow at the Leverhulme Centre for the Future of Intelligence.
Links aren’t working on Twitter due to an “internal change” that had “some unintended consequences.”
If you were accustomed to a time when Twitter — while far from perfect — was a place where you could dependably digest a wide range of breaking news, politics, celebrity gossip, or personal musings, it’s time to accept a new reality.
Twitter is becoming a degraded product.
In the four months since Elon Musk took over the company, the app has experienced major glitches — such as when, on Monday, all links to external websites stopped working. (Twitter has acknowledged the error, posting on a company account that “parts of Twitter are not working as expected” due to “an internal change that had some unintended consequences,” and that it’s trying to fix the problem.) Twitter’s social media dashboard application that’s popular with super users, Tweetdeck, also appeared to be down. It’s not the first time: Last month, users around the world couldn’t post tweets, send messages, or follow new accounts for several hours. While Twitter, like other social media networks, has always had periodic outages, under Musk, the app’s unpredictability isn’t just limited to technical issues. Musk’s erratic decisions are degrading the integrity of Twitter’s core product and alienating wide swaths of users.
Musk’s Super Bowl meltdown, as reported by Platformer, is one of the clearest signs so far of Twitter’s decline. Musk, apparently livid because his tweets about the Super Bowl were getting fewer views than President Joe Biden’s, flew to Twitter’s headquarters and ordered engineers to change the algorithm underlying Twitter’s main product to boost his own tweets above everyone else’s so that they show at the top of Twitter users’ “For You” page. Musk’s cousin, James Musk — who is now a full-time employee and a reported “fixer type” within the company — reportedly sent an urgent 2 am message asking all capable engineers to help, and the company tasked 80 engineers to manually tweak Twitter’s underlying system to promote Musk’s tweets.
Soon after the change, many users started noticing their feeds had been bombarded with Musk’s tweets. Musk seemed to acknowledge the phenomenon, posting a meme showing a woman labeled “Elon’s tweets” force-feeding a bottle of milk to another woman labeled “Twitter,” and later posting that Twitter was making “adjustments” to its algorithm.
The episode demonstrates how Twitter has become less and less dependable. The platform’s basic product design is now tailored to the whims of Musk, a leader who seems to prioritize his own image and “free speech absolutist” ideology above business interests.
A few examples: Musk, in the free-speech spirit of letting people say almost anything they want on Twitter, restored the accounts of thousands of previously suspended users, including neo-Nazi and QAnon accounts. That was one of the driving factors, researchers told the New York Times, behind a rise in hate speech on the platform, including an over 200 percent increase in anti-Black slurs from when Musk took over until December 2022 — upsetting many users who already struggled with harassment on the platform.
On the product front, Musk has rushed projects that have caused chaos on the platform. Musk’s most high-profile product, Twitter Blue, a paid version of the app that let anyone buy a verification checkmark badge, had a disastrous initial rollout. Musk — who has long beefed with the mainstream press — framed Twitter Blue as a way to take away the special privileges, such as checkmarks, that “elites” like journalists had on the platform, unless they paid up. But the poorly thought-out changes to Twitter’s verification policy ended up flooding the platform with spam, as newly verified accounts used their checkmarks to convincingly impersonate public figures, including Musk. The release was pulled back and delayed twice before finally coming out in December.
Under Musk, Twitter also recently blocked third-party apps that improved people’s experience on the app, like Tweetbot. While Twitter is promising developers a revamped paid version of its API, the way Twitter suddenly cut off access has soured its relationship with outside programmers whose add-on apps enriched the site.
Since Musk has laid off or fired more than half of Twitter’s staff, the people left to clean up the mess are short-handed. That includes teams that deal with fixing bugs, content moderation, and courting advertisers.
When Elon Musk first bought Twitter, even though many were skeptical about the billionaire, there was also some optimism that Musk could turn the company around. Investors hoped that Musk, the prolific and successful entrepreneur, could revive a company that was unprofitable and seen as not living up to its full business potential. Musk’s ideological supporters saw him, a self-appointed “free speech absolutist,” as someone who could make Twitter less restrictive and open to a wider range of speech.
Now we’re seeing Musk’s potential to improve Twitter — on the business and ideological fronts — unrealized.
On the business side, Twitter’s main line of revenue is in jeopardy as 500 big-name advertisers have paused spending on the platform since Musk took over, in large part over concerns about Musk’s overall erratic behavior and the rise in what researchers say is an “unprecedented” rise in hate speech on the platform. Twitter’s top 30 advertisers dropped their spending on Twitter by an average of 42 percent from when Musk took over until the end of 2022, according to Reuters. Musk’s solution to Twitter’s loss of advertiser dollars is to get more people to pay for Twitter, but that doesn’t seem to be working so far. Twitter only has around 180,000 people in the US who are paying for subscriptions to Twitter as of mid-January 2023, or less than 0.2 percent of monthly active users, according to a recent report by the Information.
While Musk claimed in November that Twitter’s user base is bigger than ever, outside data contradicts that claim. According to the data intelligence firm SimilarWeb, Twitter actually had higher traffic in March 2022 — before Musk took over — than it does now, and Twitter saw the growth in the number of visitors decline year over year from 4.7 percent in November 2022, when Musk took over, to -2 percent in Jan 2023.
On an ideological front, Musk’s Twitter has failed to live up to its free speech standards time and time again, starting with Musk suspending comedians like Kathy Griffin (who made fun of him) and barring users from talking on the platform about Twitter’s competitors, like decentralized social network Mastodon (after a flurry of criticism, Musk reversed the policy).
Even some popular figures who supported Musk for his free speech stance, like independent journalist Bari Weiss, have recanted their support after Musk banned several prominent journalists who have criticized him (Musk argued that the journalists doxxed him, which they denied). In recent months, former Twitter CEO and co-founder Jack Dorsey, who in April endorsed Musk as his successor and said he is the “singular solution” he trusts to run Twitter and “extend the light of consciousness,” has also shifted his stance and started to openly criticize Musk’s leadership, including all the recent technical glitches.
The main group of people who seem to steadfastly support the new Twitter is conservative figures and politicians. After Musk granted amnesty to many suspended accounts of right-wing provocateurs and political leaders, including shock jock Andrew Tate, Rep. Marjorie Taylor Greene (R-GA), and former President Donald Trump, Musk has achieved hero status in right-wing circles, and has even had Republican-led legislation drafted in his name that would require the Department of Justice to disclose money it spends on Big Tech companies. Musk has also earned conservative admiration for his work to uncover examples of alleged liberal bias in Twitter’s old guard, most prominently with the “Twitter Files,” a series of documents showing how Twitter made decisions about its content policies with input, at times, from US politicians and government agencies.
Even if Musk’s conservative fans love how he’s running Twitter, if the app is glitchy and more users leave the platform altogether, it won’t be of much use to them anymore. Nor will it be for Musk, who needs a healthy, money-making app in order to pay back some $13 billion he borrowed from creditors to buy Twitter.
Update, March 6, 1:25 pm: This story was originally published on February 16 and has been updated to include Twitter’s issue with external links.
The science fiction pioneer on making a template for Mark Zuckerberg, not making movies, and a worrisome climate change scenario.
Every science fiction author tries to imagine the future. But very few get what Neal Stephenson is experiencing: Some of the world’s most powerful companies are actively trying to create the future he sketched out three decades ago.
That would be in Snow Crash, the 1992 dystopia/parody he wrote about people who escape the physical world by strapping on goggles and disappearing in the metaverse. Which is now the vision of the world Mark Zuckerberg is actively embracing, both by burning billions on the effort and renaming his company Meta. Apple is also chasing after this idea, and may finally unveil a new headset to make it happen this spring. Microsoft has made a stab at this too — as has Stephenson himself, when he went to work for the hyped-but-fizzled Magic Leap augmented reality startup.
Even if the real-world metaverse doesn’t pan out, Stephenson has had an enormous influence on how we think about tech today. People who’ve never written a line of code love his books — and so do bona fide nerds, like the Google Earth developers who used Snow Crash as inspiration, or Amazon founder Jeff Bezos, who hired Stephenson to work on his Blue Origin rocket startup.
I talked to Stephenson about Snow Crash’s legacy — some of which got auctioned off this week at a Sotheby’s auction — and much more for the Recode Media podcast. We discussed whether the metaverse can exist even if high-end virtual reality goggles never catch on; why he’s never been able to turn his work into a movie, TV show, or game; and his fear of a looming ecological disaster and the science he thinks could solve it.
Here’s an edited excerpt from our chat:
“Holy shit. You know, maybe people are actually taking this seriously.”
Peter Kafka
When did you get a sense that the Jeff Bezoses and Mark Zuckerbergs of the world were really influenced by Snow Crash? That this was really meaningful to tech people who were building things?
Neal Stephenson
I started hearing about it in the mid-90s, as the Internet became a thing. I was on the WELL, which is an early BBS, and there were a lot of tech people there. And so I started getting the idea that it was well-received. I started to hear from people in the tech industry who were reading it, and it gradually became clear. When Google Earth came out [in 2001], word reached me through the grapevine that the Earth application described in Snow Crash had been somewhat inspirational for that. So at that point I was like: “Okay, holy shit. You know, maybe people are actually taking this seriously. “
Peter Kafka
And then cut to [2021] where Zuckerberg renames his company Meta and says, I want to build the metaverse and spend billions of dollars. Did he reach out to you prior to that?
Neal Stephenson
No. And not after either. So there’s been zero communication.
Peter Kafka
Your book, like a lot of science fiction, is describing a dystopia. And it struck a lot of people, including me, as weird that a consumer company, one of the biggest companies in the world with 2 billion users, would say, “This is the future we’re pivoting toward.” What do you make of that?
Neal Stephenson
So, a couple of things. One is, Snow Crash is a dystopian novel, but it’s also kind of a parody of dystopian novels because even then …
Peter Kafka
The main character’s name is “Hiro Protagonist.”
Neal Stephenson
Yeah. And, you know, there had been enough of that kind of literature out there that the tropes had become familiar. And just rehashing them without any self-awareness or humor would have been a little weird. So there’s that. And then the world — the real world — certainly has got its dystopian aspects in that book. But the metaverse itself, I think, is kind of neutral. The first parts of it that we see are kind of garish. And people are playing violent games and there’s lots of ads and tacky crud there. It’s the first thing that meets the eye when you go into the metaverse. But it’s also made clear that there are people like Hiro and Ng who have put a huge amount of effort into making extraordinarily beautiful, detailed houses that they can live in in the metaverse.
Peter Kafka
To me, the striking thing is not so much the metaverse is dystopian but that it’s built to escape a world that is dystopian. We’ve seen that in a bunch of novels. And it just seems like a weird thing to say, “This is the future, we think this is great,” because it implies that the rest of the world is going to fall apart.
Neal Stephenson
Yeah, you’d have to ask him.
“My theory is that a witch placed a curse on me”
Peter Kafka
You previously said you’re “interested in game engines as cultural media for new creative work.” So should I assume that there will be a Neal Stephenson game that I’m going to play at some point?
Neal Stephenson
I’m trying to build something like that. There’s a lot of hoops to jump through first involving rights and financing that are very boring to talk about.
Peter Kafka
Not for me. I nerd out on that stuff.
Neal Stephenson
That’s your deal? Well, it’s part of what we’re calling “the extended Snow Crash universe timeline,” which is sequel/prequel material, basically, to Snow Crash.
Peter Kafka
It sounds like you may not have the rights to make your own book into something.
Neal Stephenson
The rights to the original book are currently controlled by Paramount.
Peter Kafka
Why hasn’t any of your work been turned into a movie, television show, or game? Especially over the last few years when there was so much money being thrown at stuff people could put on streamers. Your work means a lot to a lot of people. It’s established IP. Why hasn’t there been a Neal Stephenson work that I’ve been able to play or watch?
Neal Stephenson
My theory is that a witch placed a curse on me. That’s the current going theory. My producing partner and I refer to what you’ve just described as “the curse.” We’ve been working on trying to break the curse. Currently the leading contender is that there’s some work underway to adapt a book I co-wrote called The Rise and Fall of D.O.D.O. into a television series. It’s still in the early stages, so it’s got a lot of hoops to jump through.
Peter Kafka
But you’ve worked for Jeff Bezos. None of the richest people in the world ever said, “I’m just going to set this up for you. I’m such a mega fan, I’m just going to open up my pocketbook and we’re going to make this thing happen”?
Neal Stephenson
That sounds like a great plan. I like that plan. When you try to implement that plan, sometimes some complications can arise, which again, I can’t get into right now. But, you want smart money. You want people who actually know how to put all the pieces together and produce something. And it is a complicated industry.
Peter Kafka
I was wondering if you were going to say, “Look, up until recently, it’s been impossible technically to make the stuff that I’ve written into something visual or a game, and I didn’t want to do a half-assed version.” I would hate to have seen what a 1997 version of the metaverse looked like.
Neal Stephenson
I’ve had that thought a lot of times. There but for the grace of God.
If someone had done an adaptation of Snow Crash in 1995, they would have said, “What’s the coolest snazzy computer graphics we can get right now and we’ll have that be the metaverse.” And then five years later, people would be looking at it like, “Oh my God, they used to think that was cool-looking.” And I’ve had a few conversations over the decades with people who were investigating adapting Snow Crash and their ideas have changed over time. And at a certain point it flipped over and it became, you know, “We’ll just shoot everything on film because the metaverse would be film-quality graphics for sure. And then we’ll manipulate it, we’ll add digital tweaks, to make it clear that this is the metaverse and not the real world.”
Peter Kafka
How much does it bother you that this has not happened?
Neal Stephenson
You know, be careful what you wish for, I guess. It’s sometimes better to have the aspiration of something than to face some of the compromises that may happen when it really materializes. But I don’t lose sleep over it because I can still write novels.
It’s much more frustrating if you’re a film director or a screenwriter and you can’t get stuff made. You need other people to mobilize huge amounts of capital to make that real. There’s a weird way in which novelists — even broke novelists — have got a kind of status in that world that is very high status because they have creative control.
Peter Kafka
You make the thing exactly the way you want it to be.
Neal Stephenson
Yeah. I can remember, way back in the ’80s, I was talking to screenwriters who had been hired to adapt some of my work. They’re driving Porsches around Beverly Hills. I’m starving. But they would come to me and say, “How did you become a writer? How could I become a novelist?” Because in their mind, status isn’t money, it’s creative control. And they wanted that kind of status.
Peter Kafka
You can’t pay your rent with status.
Neal Stephenson
Yeah. Well, that’s true.
“The metaverse initially is going to be experienced by almost everyone on a flat screen”
Peter Kafka
Do you think in the future — assuming the tech gets there and assuming there’s a reason to use it, which are both huge things — that humans are going to want to wear AR/VR goggles? I went to the new Avatar a couple months ago and it’s a three-hour movie and I was seeing it in IMAX with the [3-D] headset on. And an hour and a half in, I was like, “I don’t want to wear these goggles anymore.”
Neal Stephenson
You hit your limit. There’s a semantic distinction between glasses and goggles. Lots of people wear glasses all day and nobody thinks twice about it. Very few people wear goggles all day. You go skiing, maybe you’ll put on goggles. Fighter pilots wear goggles, but goggles are not generally a long-term wear kind of item. And no matter how good the experience is, wearing that stuff for, as you say, more than 45 minutes or an hour is not enjoyable for a lot of people.
On the other hand, the game industry has taught everyone to experience 3D worlds through a rectangle — a flat rectangular screen — and it works great. You’re using your keyboard and your mouse or whatever your control system is. And people just fluently pick that up and they’ll play that for hours. So I think that goggles are going to be a thing. I like goggles. I know people who make goggles of various types, and I can’t wait to see what comes out of that industry. But I think that most people are going to continue experiencing 3D worlds most of the time through screens.
Peter Kafka
And does the metaverse work if it’s a flat-screen experience for those people?
Neal Stephenson
Totally. I keep forgetting to mention this but in my view, the metaverse initially is going to be experienced by almost everyone on a flat screen.
Peter Kafka
A television set or iPhone.
Neal Stephenson
Yeah, some version of that. Because that’s just that’s the reality. That’s what the market is.
“The only things worth talking about right now are carbon and the fracturing of society by social media”
Peter Kafka
You imagine the future for a living. Are you optimistic or pessimistic?
Neal Stephenson
So I think that the only two things worth talking about right now are carbon and the fracturing of society by social media. They’re both equally concerning. I don’t know what to do about social media. I’m not a people person, in a lot of ways. So I tend to think about carbon. I’ve been thinking a lot about carbon, carbon sequestration in particular, geoengineering, all that stuff.
Peter Kafka
Getting the carbon emissions out of there.
Neal Stephenson
How do we reduce carbon emissions and remove the hundreds of billions of kilograms of carbon that we’ve already put into the air? I think we’ll beat that problem. But I think it’s going to be the biggest engineering project in human history. It’s going to transform the world — the built environment — because we simply can’t do it without doing engineering on a massive scale. I think we’ll succeed at it. But we’ll have some bad times between now and then.
I think we’ll start to see the kinds of mass casualty events that are described in Kim Stanley Robinson’s book, The Ministry for the Future, where you might see millions of people dying of heat stroke in a certain area over a very short period of time. When the temperature goes up, the humidity goes up, the power goes out. And when that kind of stuff starts happening — which I sadly think it will in the next decade — it’s going to have incredibly powerful political ramifications.
Peter Kafka
I was going to say we’re ending this [conversation] with cautious optimism, but I don’t know if I can call it that.
Neal Stephenson
I hope that stuff doesn’t happen, but I think even the threat of it is going to lead, eventually, to people taking action.